Home

TM-ngram-partition

TM-ngram-partition[map, max-ngram-len] input-seq
TM-ngram-partition[map, max-ngram-len, merge-str, seq-merge-str] input-seq
This is a member of the "template machine" collection of operators
So far, they are just experimental, and further testing is required
This one breaks a sequence up into a sequence of ngrams
But only on ngrams that are known with respect to some operator (RULE NORMAL only), in this case "map"
Single elements are considered "known"
max-ngram-len tells us the max size ngrams to consider in this process
The elements inside the ngram are merged using "merge-str"
The sequences of ngrams are merged using "seq-merge-str"
If not specified they default to " " and " . " respectively
The output of this operator is the superposition of the results

-- define a pair of known ngrams:
-- noting the ngrams can overlap each other:
map |A B C> => |yes>
map |B C D> => |yes>
-- now invoke it:
TM-ngram-partition[map, 3] ssplit[" "] |A B C D E>
    |A . B . C . D . E> + |A . B C D . E> + |A B C . D . E>

-- now learn another ngram:
map |D E> => |yes>
-- now invoke it:
TM-ngram-partition[map, 3] ssplit[" "] |A B C D E>
    |A . B . C . D . E> + |A . B . C . D E> + |A . B C D . E> + |A B C . D . E> + |A B C . D E>

TM-generate

sequence compound context